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A GENERAL  TECHNIQUE  FOR 
R & D COST  FORECASTING 

INTRODUCTION 

During  the  1920*s,  two  statisticians,  Raymond  Pearl  and  L.J. 
Reed,  were  extensively  involved  In  research  dealing  with  forecasting 
rates  of  population  and  biological  growth.  They  discovered  that  the 
S-shaped  curve  form  of  Figure  1 accurately  represented  growth  of 
these  types  which  are  characterized  by  three  distinct  phases:  (1)  a 
slow  period  of  development,  (2)  a rapid  period  of  expansion,  and 
(3)  a tapering  off  at  maturity, 

, 6HoivrH 


The  growth  rates  of  a number  of  developing  industries  were  also 
found  to  follow  this  general  format,  but  most  attempts  to 
use  this  curve  form  for  forecasting  were  non-productive.  The 
failure  of  the  two  forms  of  the  S-shaped  curve  (the  Logistic  and 
the  Gompertz) (See  Appendix  1)  to  provide  accurate  forecasts  can  be 
attributed  to  two  principle  causes; 

(1)  The  equation  for  the  Gompertz  curve  assumes  symmetry 
around  an  inflection  point  which  is  the  geometric  mean  of  the  Y 
values,  while  the  equation  for  the  Logistics  curve  makes  the  same 
assumption  for  the  true  mean  of  the  Y values.  Neither  of  these 
assumptions  Is  usually  justified  In  real-world  situations.^ 

(2)  Because  of  the  order  of  the  polynomials  needed  to  express 
these  curves,  slight  perturbations  of  the  data  early  in  the  growth 
process  can  quickly  force  the  equation  for  either  of  the  curves  to 
forecast  an  unrealistically  high  or  low  figure  for  total  growth. 

This  failure  to  forecast  accurately  becomes  important  in  the 
context  of  Research  and  Development  within  the  Department  of  Defense 
because  R & D programs  also  follow  the  S-shaped  curve  form  in  terms  of 
both  cumulative  dollar  expenditures  and  cumulative  milestone  com- 
pletion over  time.  This  fact  should  be  no  surprise  since  the 
R & D process  Is  Intuitively  a biological  growth  process  In  which 
there  Is  slow  Initial  development  of  the  specific  project,  followed 
by  a fairly  rapid  building  phase  which,  in  turn.  Is  followed  by  a 
taperlng-off  to  completion. 

^Tablc  VI  shows  the  actual  Inflection  point  for  21  weapon  systems. 
Note  that  In  no  case  does  this  point  consistently  fall  anywhere  close 
to  either  the  true  or  the  geometric  mean. 
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The  existence  of  these  three  phases  in  Department  of  Defense 
R & D programs  was  verified  by  taking  the  Cost  Performance  Reports 
(CPRs)  for  22  different  weapon  and  component  systems  and  tracing 
the  actual  costs  as  they  were  Incurred  throughout  the  life  of  each 
R & D program.  Without  exception  the  S-sbaped  curve  was  the 
dominating  factor  in  the  system  development  actually 

occurred.  The  milestones  associated  with  these  programs  also 
exhibited  the  same  characteristics  in  every  case  for  which  data 
were  available. 

The  results  based  on  the  original  22  weapon  systems  were 
then  checked  against  new  and  very  complete  data  which  were  provided 
by  another  source  on  15  weapon  systents.  Again  the  connection 
between  R & D expenditures,  the  S-shaped  curve,  and  milestone 
completion  were  firmly  established. 

Conventional  wisdom  would  suggest  that  since  the  Logistics 
and  Gompertz  curve  forms  are  generally  non-productive  forecasting 
tools,  one  should  proceed  instead  to  construct  an  econometric 
model  of  the  R & D process.  This  model,  which  would  embody  each 
of  the  endogenous  variables  In  any  given  R & D project,  could 
then  use  the  historical  cost  data  for  each  of  these  Inputs  (labor, 
raw  materials,  etc.)  to  establish  trends.  The  trends  could  then 
be  coai>lned  to  provide  an  overall  forecast  of  the  final  cost  of 
the  R & D project. 
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Unfortunately,  the  existing  data  concerning  the  prices  of 
many  raw  materials  such  as  aluminum,  stainless  steel,  titanium, 
steel  alloys,  etc.,  are  either  misleading,  erroneous,  or  simply 
do  not  reflect  the  true  prices.  In  addition,  labor  data  are 
often  not  available  for  the  narrow  categories  of  labor  employed  • 
in  a given  K & D project,  nor  do  they  properly  reflect  local 
or  regional  conditions  which  may  significantly  affect  wage 
rates. 

These  deficiencies  rule  out  the  use  of  conventional  econometric 
modeling  techniques,  and  one  is  faced  with  the  problem  of  finding 
some  known  element  common  to  all  R & D programs  which  could  be 
adapted  as  a forecasting  tool. 

Since  the  S-shaped  curve  is  a common  element  in  all  R & D 
projects,  this  paper  will  deal  with  an  effort  to  modify  the  curve 
In  such  a way  that  accurate  forecasting  can  be  achieved. 


CHAPTER  I 
FITTING  THE  CURVE 

The  major  problem  in  handling  an  S-shaped  growth  curve  is  one 
of  keeping  under  control  the  higher-order  polynomials  necessary 
to  express  this  complicated  curve  form.  If  one  realizes  that  any 
S-shaped  curve  is  merely  the  cumulative  form  of  a bell  curve  (which 
may  or  may  not  be  skewed)  as  shown  in  Figure  2,  a solution  to  this 
particular  problem  becomes  apparent. 


Figure  2.  The  Derivation  of  the  S-sheped  Curve 
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The  S-shaped  curve  may  be  separated  (or  "broken")  at  the 

inflection  point  to  yield  two  simpler  curves  of  a form  which  may 

be  expressed  by  either  a logrithmic  (y  = ax^)  or  a quadratic 

2 

(y  = a + Xj^  + b^  Xj^)  equation.  Standard  econometric  fitting 
techniques  may  then  be  utilized  to  determine  which  of  these  two 
curve  forms  is  the  best  fit. 

However,  selecting  the  best  curve  form  is  greatly  com- 
plicated by  the  fact  that  the  time  dependent  nature  of  these 
curves,  and  indeed  the  entire  R & D process,  tends  to  cause  a 
great  deal  of  autocorrelation  in  the  resulting  data.  This  auto- 
correlation, which  is  a violation  of  the  necessary  assumption  that 
regression  residuals  are  not  related,  must  be  dealt  with  and 

eliminated  from  the  data  before  a valid  selection  of  equatioiis  can 
* 

be  made  to  express  the  lower  and  upper  halves  of  the  S-shaped  curve. 
For  this  particular  research,  autocorrelation  was  removed  through 
the  use  of  Generalized  Least  Squares  following  an  autoregressive 
transformation  which  used  a two-step  procedure  for  the  estimation 
of  Rho.  This  process  is  explained  in  its  entirety  in  Appendix  2. 

It  should  be  noted  that  the  treatment  of  each  additional 
order  of  autocorrelation  by  this  method  removes  one  additional 
data  point  from  the  lower  end  of  whichever  half  of  the  S-shaped 
curve  is  being  considered.  This  occurs  because  each  succeeding 
transformation  steps  back  in  time  one  data  point  further  in  an 
attempt  to  remove  any  past  Influences  on  the  current  regression 
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residual.  This  process  has  an  inherent  benefit  in  that  it  tends 
to  weight  recent  observations  more  heavily  than  past  observa- 
tions. The  result  is  a rather  sophisticated  smoothing  technique 
which  the  user  may  vary  to  assure  that  the  fitted  curves  will 
converge  rapidly  on  the  proper  values. 

Tables  1 through  IV  show  the  results  of  this  curve  fitting 
technique  when  it  was  applied  to  both  actual  and  budgeted  cost 
figures  for  22  major  weapon  systems  or  R & D projects.  In  almost 
every  case,  the  quadratic  equation  provided  the  best  fit  of  both 
the  upper  and  lower  segments  of  the  S-shaped  curve.  The  statistics 
accompanying  each  of  the  curve  equations  indicate  that  the  upper 
and  lower  curve  segments,  when  rejoined,  do  provide  an  excellent 
proxy  for  the  original  S-shaped  curve  for  each  R & D project,  and 
the  mean  square  error  figures  accompanying  each  equation  assure 
that  these  reconstructed  S-shaped  curves  are  useful  forecasting 
tools. 

With  the  curves  for  each  of  the  specific  programs  developed, 
the  same  technique  of  calculating  the  upper  and  lower  halves  of  the 
S-shaped  curve  and  then  macing  these  segments  at  the  Inflection 
point  can  be  used  to  derive  a general  curve  based  on  all  weapon 
systems  for  which  data  are  available.  This  general  curve  can  then 
serve  as  the  forecasting  vehicle  for  new  weapon  systems. 

The  actual  calculations  for  the  general  curve  are  accomplished 
by  pooling  all  of  Che  data  for  17  systems.  Table  VI  shows  that  the 
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mean  inflection  point  for  these  systems  occurs  at  the  56.2%  of 
expenditures  and  46.2%  of  time  points  with  the  width  of  one  standard 
deviation  around  this  point  being  5.4%  for  expenditures  and  7.3% 
for  time.  Considering  that  these  figures  were  derived  from  a 
collection  of  systems  which  were  so  diverse  as  to  include  an  early 
1950's  era  weapon  system  (the  F-105) , cargo  and  bomber  aircraft  (C-5, 
C-141,  B-1,  etc.),  engines  and  guns  (A-10) , and  future  projects 
such  as  space  tugs,  these  figures  represent  ^ surprisingly  narrow 
confidence  interval. 

Exhibits  I and  II  show  the  results  of  fitting  the  curve 
segments  using  a quadratic  equation  for  both  the  lower  and  upper 
halves.  A certain  degree  of  heteroskedasticity  was  Introduced 
into  these  data  sets  by  a normalization  of  the  data  which  forced 
the  S-shaped  curve  to  begin  at  the  0%  time  and  expenditures  point 
and  end  at  the  100%  time  and  expenditures  point.  Because  of  the 
obvious  significance  of  the  data  provided  in  Exhibits  1 and  II, 
the  heteroskedasticity  was  not  deemed  to  be  a significant  problem. 

Figure  3 shows  the  general  curve  form  and  a lo  confidence 
Interval  around  this  curve.  The  calculations  for  this  confidence 
interval  may  be  found  in  Table  VI.  This  type  of  curve  derivation 
has  the  obvious  advantages  of  generating  a result  which  should  have 
wide  application  across  the  broad  spectrum  of  weapon  systems,  and 
as  such  it  will  form  the  basis  for  the  forecasting  efforts  which 
will  be  explored  In  the  next  chapters. 
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CHAPTER  II 

THE  RATIONALE  BEHIND  THE  USE 
OF  THE  S-SHAPED  CURVE 

Chapter  I showed  that  the  S-shaped  curve  described  every 
military  R & D project  which  was  investigated,  and  Chapter  II 
will  demonstrate  a practical  method  of  finding  the  curve  for  either 
a specific  project  or  a general  model.  However,  the  rationale 
behind  the  use  of  both  a two-part  curve  and  the  initial  budget 
data  in  the  forecasting  process  deserves  further  explanation. 

Since  no  one  can  estimate  the  cost  of  developing  a new  weapon 
system  without  an  effort  which  would  virtually  duplicate  the 
original  contract-letting  process.  It  Is  obvious  that  the  rational 
approach  Is  to  accept  the  R & D budget  proposed  by  the  contractor 
as  the  best  initial  estimate  of  the  project  cost. 

However,  once  this  approach  Is  decided  upon  and  It  Is  realized 
that  a two-part  S-shaped  growth  curve  can  be  chosen  to  express  the 
program  as  a whole,  three  very  interesting  approaches  may  be 
adopted: 


(1)  The  proposed  project  budget  may  be  compared  to  all  past 
R & D budgets  to  see  If  the  expenditure  pattern  Is  generally  rational 
In  light  of  R & D experience  with  other  weapon  systems.  Figure  3 
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shows  the  general  curve  with  a lo  confidence  band  based  on  the  R & D 
projects  which  were  studied  for  this  paper.  Considering  the  diversity 
of  the  weapon  systems  which  were  Investigated,  this  band  represents 
a fairly  narrow  range  within  which  the  expenditure  pattern  for  any 
new  R & D project  should  be  expected  to  fall.  If  the  proposed 
expenditure  pattern  falls  outside  of  this  range,  the  contractor 
should  be  expected  to  explain  why  his  program  is  unique  in  this 
respect . 

(2)  The  proposed  budget  expenditure  pattern  can  be  compared 
to  the  proposed  project  milestones  to  check  the  specific  rationality 
of  the  budget  proposal.  Table  V shows  a reperesentative  selection 
of  projects  for  which  milestone  data  are  available.  There  is  a very 
high  order  of  correlation  between  the  budget  expenditure  pattern 
and  the  pattern  in  which  the  milestones  are  accomplished.  This 
suggests  that  the  manner  in  which  the  milestones  are  completed  over 
time  provides  an  excellent  cross-check  on  the  way  in  which  the  budget 
should  be  expended.  And  it  also  means  that  slips  or  changes  in  the 
order  of  milestone  completion  will  invariably  be  reflected  in  the 
expenditures  for  the  entire  project. 

(3)  When  the  contractor  or  procurement  officer  considers 
uncertainty  in  any  R & D program,  he  can  generally  resolve  this 
uncertainty  into  different  types  of  unknowns.  Drake  states  that 

these  unknowns  are  of  two  kinds: 

(1)  The  unknowns  that  he  is  aware  of  and 
believes  he  can  resolve  when  he  accepts  a 
contract — for  example,  the  configuration  of 
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an  aircraft's  slats,  flaps,  thrust  reversers, 
speed  brakes,  and  other  devices  needed  to  meet 
the  specified  performance  factors  (take-off, 
landing  distance,  etc.)- 

(2)  The  unknowns  which  are  bound  to  crop  up 
unexpectedly  and  for  which  he  Is  not  prepared — 
the  'unanticipated  unknou-ns.'  Strictly  speaking, 
they  are  the  unknowns  that  aannot  be  foreseen.^ 


In  other  words , type  1 unknown  are  known  unknowns  while  type  2 
unknowns  are  unknown  unknowns. 


If  one  then  pictures  himself  standing  at  the  beginning  of  an 


R & D project  and  looking  do%m  the  time  line  toward  the  completion 
of  the  pr  ject,  he  will  see  the  two  different  types  of  uncertainty 
shown  In  Figures  4 and  5. 

oHcttrr/HHTy 
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Figure  4.  Uncertainty  Due  to  Known  Unknowns 


^Drake,  Hudson  B.,  "Major  DoD  Procurements  at  War  with  Reality," 
The  Harvard  Business  Review.  January-February  1970,  p.  124. 
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Figure  5.  Uncertainty  Due  to  Unknown  Unknowns 

These  two  figures  can  be  combined  to  show  that  the  total 

program  uncertainty  is  really  a kinked  curve  as  shown  in  Figure  6 
wucEflTffiHrV 


The  critical  point  is  the  inflection  point  of  the  program  since  it 
is  at  this  point  that  most  of  the  uncertainty  has  been  dispelled. 
Thus,  accurate  forecasting  is  most  essential  and  also  most  difficult 
in  the  early  part  of  the  R & D program.  The  two-part  curve  develop- 
ment overcomes  the  problem  of  changing  uncertainty  by  allowing  the 
researcher  to  forecast  a new  lower  curve  and  inflection  point  from 
the  actual  expenditures  in  the  program  being  investigated.  This 
forecast  may  then  be  constrained  by  using  the  top  half  of  either 
the  budget  or  the  general  curve  in  the  area  where  uncertainty  is 
lower  and  hence,  these  curves  should  be  the  most  accurate  estimators 
of  program  performance. 

It  should  be  assumed  that  since  the  contractor  and  the 
government  have  both  attempted  to  plan  the  R & D project  in  the 
best  possible  manner  given  the  financial  and  political  constraints 
on  the  program,  this  planning  effort  should  not  be  wliolly  dis- 
regarded as  soon  as  the  first  actual  expenditures  begin  to  appear. 
Instead,  it  is  more  rational  to  let  the  general  framework  of  the 
expenditure  pattern,  as  expressed  in  the  budget  curve,  continue  to 
represent  this  planning  as  an  inherent  constraint  during  any  fore- 
casting activity.  Used  in  this  manner,  the  original  budget  expen- 
diture curve  can  be  viewed  as  a storehouse  of  subjective  or  judg- 
mental information  which  may  be  used  to  constrain  a forecast  and 
thus  allow  for  future  impute  by  the  managers  in  control  of  the 
R & D program. 
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These  three  points  Indicate  why  the  use  of  the  two  part 
S-shap  'urve  Is  particularly  appealing.  Not  only  is  the  contractor 
subject'*'  CO  Intense  scrutiny  during  the  initial  planning  phase 
(points  one  and  two) , but  once  he  passes  these  checks  his  planning 
may  then  be  used  as  a forecasting  constraint  during  the  rest  of 
I the  program. 
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CHAPTER  III 

DEVELOPING  AliD  USING  THE  S-SHAPED  CURVE 
The  general  method  for  the  development  and  use  of  the  S-shaped 
curve  for  a specific  R & D program  follows  these  steps: 

I.  The  budget  figures  for  the  R & D program  are  gathered 
and  the  monthly,  quarterly,  or  other  Incremental  expenditures  are 
recorded  as  a cumulative  percentage  of  the  total  expenditure. 
Similarly,  the  amount  of  time  over  which  tha  program  Is  to  run  Is 
determined  and  each  succeeding  Increment  Is  recorded  as  a cumulative 
percent  of  the  total  program  time.  This  step  has  the  effect  of 
normalizing  the  program  so  that  It  can  be  compared  with  all  past 
R & D programs  when  plotted  on  the  axis  of  Figure  7. 
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II.  with  the  data  arrayed  In  this  normalized  format  and 
plotted  on  the  axis  of  Figure  7,  the  budget  expenditure  pattern  may 
be  Immediately  checked  for  general  rationality.  This  Is  accomplished 


by  determining  whether  or  not  the  proposed  budget  curve  falls 
within  the  confidence  band  which  has  been  constructed  for  all 
past  R & D projects  (Figure  3).  If  the  proposed  budget  expenditure 
pattern  falls  within  this  band,  the  R & D project  planning  Is  In 
accordance  with  past  R & D experience.  However,  if  the  expenditure 
pattern  falls  outside  the  confidence  band  It  should  be  determined 
why  this  particular  program  is  planned  to  proceed  differently  from 
all  past  programs  have  proceeded. 

III.  The  total  number  of  milestones  connected  with  all  phases 
of  the  R & D project  Is  determined  and  the  month  In  which  the  end 
or  completion  point  of  each  milestone  occurs  is  noted.  The  cumula- 
tive percentage  of  completed  milestones  is  then  calculated  for  each 
period  for  which  budget  data  are  available.  This  step  has  the  effect 
of  putting  milestone  accomplishment  In  the  same  normalized  form  as 
the  financial  data.^  These  normalized  milestone  accomplishment  data 

^At  first  It  may  seem  counterintuitive  to  sum  up  milestones 
In  the  same  manner  In  which  one  sums  up  dollars.  However,  even 
though  it  Is  obvious  that  no  two  milestones  represent  Identical 
amounts  of  accomplishment,  deriving  cumulative  sums  of  milestone 
completions  by  financial  period  has  Invariably  provided  an  excellent 
picture  of  the  manner  In  which  the  program  is  expected  to  proceed. 

It  should  be  noted  that  the  best  results  have  been  obtained  when 
the  milestones  were  expressed  on  the  lowest,  or  most  specific  level 
available.  Every  attempt  should  be  made  In  this  process  to  avoid 
double  counting  between  specific  and  general  program  milestones,  al- 
though no  adverse  effects  have  been  noted  when  different  specific  and 
general  milestones  were  sumsMed  together  using  this  technique. 
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are  then  regressed  with  the  normalized  budget  data.  The  results  of 
this  regression  should  show  an  extremely  significant  relationship 
between  the  budget  expenditures  and  the  milestone  completion. 

Table  V shows  the  results  for  five  sample  programs  to  give  a general 
idea  of  the  significance  of  the  relationship.  If  this  relationship 
does  not  appear,  the  proposed  expenditure  pattern  for  the  specific 
program  in  question  is  highly  dubious  and  indicates  an  unrealistic 
program  plan  which  could  easily  lead  to  financial  problems. 

IV.  Next  locate  the  largest  incremental  change  in  budgeted 
expenditures  which  is  followed  by  two  periods  of  decreasing 
expenditures.  This  increment  is  designated  as  the  inflection  point. 
The  S-shaped  curve  is  broken  at  this  point  and  the  inflection  point 
becomes  the  last  data  point  in  the  first  (or  lower)  curve  and  the 
first  data  point  on  the  second  (upper)  curve.  This  common  point 
allows  the  curves  to  be  spliced  again  after  curve  fitting.  Table  VI 
shows  the  location  of  the  inflection  points  for  the  R & D programs 
included  in  the  initial  part  of  this  study.  Note  the  high  degree 

of  variability  in  the  point  locations. 

V.  Equations  for  the  lower  and  upper  portion  of  the  budget 

curve  are  developed  by  the  regression  scheme  discussed  In  Appendix  2. 

2 

Due  to  the  shape  of  the  curves,  a quadratic  (y  ••  a + b^  -f  b^  x^) 
or  logrlthmlc  (y  * ax^)  curve  equation  is  usually  appropriate. 
Particular  care  muat  be  taken  In  this  step  to  assure  that  the  curve 
equations  which  are  developed  have  dealt  with  the  problems  inherent 
In  the  use  of  time  aeries  data.  Failure  to  correct  the  problem  of 
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autocorrelation  will  result  In  curve  equations  which  are  of  little 
value  and  which  will  adversely  affect  the  performance  of  the 
completed  model.  To  avoid  this  si tuatlon,  some  techniques  of  the 
type  explained  In  the  appendix  must  be  used  at  this  Juncture,  and 
Indeed,  should  be  used  In  any  R & D forecasting. 

VI.  Once  the  curve  equations  have  been  developed  from  the 
budget  data,  two  specific  types  of  knowledge  have  been  gained: 

A.  The  equation  form  which  best  fits  the  R & D budget 
data  has  now  been  determined.  This  Is  usually  a quadratic  form  for 
both  the  upper  and  lower  halves  of  the  S-shaped  curve  and  this 
specific  curve  form  should  be  used  with  any  actuals  when  later 
attempts  are  made  to  forecast  the  end  price  of  the  R & D project. 

B.  Equations  expressing  the  subjective  planning  Inherent 
in  the  R & D program  are  now  available  for  the  upper  and  lower  parts 
of  the  S-shaped  curve.  These  original  equations  can  be  used  as 
constraints  during  forecasting,  thus  providing  a method  of 
Incorporating  this  subjective  Information  Into  the  final  price 
forecast.  The  specific  methods  by  which  the  two  part  S-shaped 
curve  may  be  used  for  price  forecasting  are  the  subjects  of  the  next 
chapter. 
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CHAPTER  IV 

THE  S-SHAPED  CURVE  AS  A FORECASTING  TOOL 
The  objective  of  the  methodology  advocated  in  the  first 
three  chapters  of  this  paper  is  obviously  the  production  of  a 
forecast  cost  for  the  system  being  developed.  In  pursuing  this 
objective  it  is  well  to  remember  Scrooge's  question  to  the  Spirit 
of  Christmas  Future:  "Are  these  the  shadows  of  things  that  will  be, 
or  are  they  the  shadows  of  things  that  may  be,  only?  Men's  courses 
will  foreshadow  certain  ends,  to  which,  if  persevered  in,  they 
must  lead.  . . . But  if  the  courses  be  departed  from  the  ends  will 
change . " 

This  quote  conveys  the  proper  manner  in  which  one  should 
employ  a forecast  to  make  it  an  effective  management  tool.  The 
program  manager  should  view  the  forecast  as  a non-threatening  means 
of  alerting  managers  to  possible  program  difficulties  and  it  should  be 
presented  not  as  a point  estimate,  but  rather  as  a range  of  values 
within  which  the  end  cost  of  the  program  is  likely  to  fall  if  the 
present  courses  of  action  are  continued.  For  the  purpose  of  this  paper 
three  points  along  this  possible  range  of  cost  will  be  identified: 
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(a)  the  best  possible  program  cost,  (b)  the  most  likely  program 
cost,  and  (c)  the  worst  possible  program  cost. 

The  best  possible  program  cost  Is  the  cost  of  the  R & D 
program  If  It  Is  assumed  that  the  second  half  of  the  program 
will  follow  exactly  the  proposed  budget  curve  Irrespective 
of  the  performance  record  established  in  the  first  lialf  of  the 
program. 

The  most  likely  program  cost  is  that  figure  obtained  if 
the  second  half  of  the  program  follws  the  course  indicated  by 
the  general  R & D curve.  This  general  curve  being  either 
one  developed  tor  all  weapon  systems,  such  as  that  curve  dis- 
played in  Figure  d,  or  a curve  developed  for  weapon  systems  of 
the  specific  type  of  the  R & D project  being  investigated.  In 
other  words,  a curve  based  on  general  missile  system  data  would  be 
used  when  the  R & D project  is  for  a missile.  Intuitively,  the 
latter  approach  should  provide  a tighter  confidence  interval  for 
the  forecast. 

The  worst  possible  program  cost  would  be  the  figure  indicated 
by  the  upper  limit  of  the  confidence  interval  around  the  forecast. 
The  second  half  of  the  general  curve  would  again  be  used,  and  the 
confidence  interval  around  this  curve  would  provide  the  upper  limit 
(within  a certain  probability)  of  the  R & D cost. 

These  three  types  of  forecasts  are  shown  in  Figure  8. 
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Figure  8.  Three  Possible  Forecasts 


from  Program  Actuals 


The  specifics  involved  in  forming  each  of  these  forecasts 

will  now  be  covered. 

THE  BEST  POSSIBLE  COST 

(1)  First  derive  the  two  halves  of  the  equation  for  the 
S-shaped  curve  in  the  manner  previously  outlined.  This  gives 
curve  1 of  Figure  9,  the  budget  curve. 

(2)  Assume  now  that  the  first  data  points  concerning 
actual  expenditure  information  have  become  available.  These  data 
points  are  first  deflated  by  dividing  the  dollar  figures  by  an 
appropriate  index.  Studies  have  shown  that  the  GNP  Deflator  is 
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usually  a good  choice  for  this  index. ^ The  deflated  figures  are 
then  converted  to  percentage  figures  by  dividing  by  the  latest 
deflated  total  program  cost,  and  these  percentage  figures  are 
plotted  on  the  axis  of  Figure  7.  This  leads  to  the  beginning  of 
an  "actuals"  curve,  shown  as  curve  2 of  Figure  9.  These  actuals 
may  be  used  to  forecast  a new  end  cost  for  the  program  as  follows: 

(a)  Derive  a new  lower  half  of  the  S-shaped  curve 
by  fitting  the  actuals  to  an  equation  of  the  form  found  to  be 
appropriate  for  the  budget  data — in  general,  this  will  be  a 
quadratic  curve. 

(b)  Using  this  quadratic  curve  equation,  insert 
the  percent  of  total  time  figure  for  the  budget  curve  inflection 
point  (35%  on  Figure  9)  to  forecast  a new  inflection  point, 

(Point  5 on  Figure  9)  and  then  use  other  points  on  the  X (time) 
axis  to  derive  a new  lower  half  for  the  S-shaped  curve. 

(c)  Now  take  the  equation  which  was  developed  for 
the  top  half  of  the  budget  curve  and  substitute  the  percent  time 
and  percent  budget  figures  for  the  forecast  inflection  point  into 
this  equation  to  calculate  a new  intercept  for  the  upper  curve. 
This  new  Intercept,  along  with  the  original  slope  figures  from  the 
budget  curve,  has  the  effect  of  "splicing"  the  equation  developed 

^Brush,  John  S.,  "Study  of  Possible  Improvements  in  the 
Accuracy  of  ^ronautlcal  Economic  Escalation  Indices,"  unpublished 
paper,  USAF  Academy,  Colorado,  February  1976. 
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from  the  first  half  actuals  to  the  budget  equation  for  the  second 
half  of  the  curve,  all  of  which  yields  the  new  S-shaped  curve  4 
of  Figure  9.  In  addition,  this  procedure  allows  the  development 
of  a forecast  for  the  end  cost  of  the  project  which  is  constrained 
by  the  planning  and  other  subjective  information  inherent  in  the 
original  budget  curve. 
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Figure  9.  The  Forecasting  Process 
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(d)  At  this  point,  one  may  take  several  different 
approaches.  First,  if  he  wants  to  learn  the  absolute  figure 
for  the  final  cost  of  the  project,  curve  4 nay  be  modified  by 
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Inclusion  of  inflation  data.  In  this  case,  the  forecast  expenditure 
data  of  curve  4 would  be  multiplied  by  an  Inflation  Index  to  get  a 
new  curve  which  is  labeled  5 on  Figure  9.  However,  in  doing  this 
one  should  have  in  mind  a concept  of  the  errors  inherent  in  any 
process  such  as  the  one  Just  described. 

Up  to  this  point  we  have  not  mentioned,  for  the  sake  of 
simplicity,  that  there  is  an  error  involved  in  forecasting  which 
should  be  expressed  as  a confidence  Interval  around  curve  4 . The 
confidence  band  indicates  that,  with  some  given  probability,  one 
may  expect  the  real  value  for  any  point  on  the  line  to  fall  somewhere 
within  this  particular  interval.  When  the  budget  curve  is  compared 
with  the  forecast  curve,  only  one  error,  the  standara  error  of  the 
forecast,  must  be  considered.  This  leads  to  the  situation  shown 
in  Figure  10. 
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Here  the  confidence  band  Indicates  the  possible  range  of  values 
(from  b to  c)  In  which  the  true,  deflated  cost  of  the  program  Is 
expected  to  fall,  and  similarly,  the  range  of  the  size  [from  ab  to 
ac]  of  the  potential  program  overrun. 

However,  If  one  desires  to  compare  the  full  cost,  with 
Inflation,  of  the  project  (5,  Figure  9)  with  the  full  Inflated 
cost  of  the  budget,  both  the  error  of  the  forecast  and  the  error 
Involved  In  developing  the  Inflation  figures  must  be  considered. 

This  has  the  effect  of  greatly  Increasing  the  size  of  the  confidence 
bands  as  Is  shown  In  Figure  11.  The  end  result  Is  that  the  ability 


to  compare  the  final  cost  of  the  project  with  the  budget  cost  is 
greatly  Impaired.  As  Figure  11  shows,  In  this  case  one  could 
anticipate  a tremendous  overrun  [a  - d]  and  an  underrun  [c  - b] 


from  the  same  data. 


I 

i 
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The  Error  of  the  Inflation  Forecast 
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The  lesson  here  Is  to  compare  figures  In  a manner  which  will 
minimize  the  errors  Involved  In  the  comparison.  In  other  words, 
the  best  picture  of  the  status  of  a project  may  be  gained  by  com- 
paring the  two  deflated  curves  shown  in  Figure  10,  This  comparison 
provides  all  of  the  Information  required  for  day-to-day  management 
of  the  program.  If  a full  end  cost  of  the  program  Is  desired,  this 
can  be  developed  quickly  by  simple  multiplication  utilizing  whatever 
Inflation  forecast  Is  deemed  appropriate  at  the  time  that  the 
information  Is  required. 

This  does  not  mean,  however,  that  one  should  not  use  the 
actual  Inflation  data  when  It  is  available.  In  this  case,  no  errors 
of  forecast  are  present  because  the  actuals  in  both  program  cost 
and  inflation  rates  are  known.  This  makes  it  very  easy  to  remove 
the  effects  of  inflation  to  see  how  much  of  an  overrun  is  actually 
attributable  to  other  causes. 

Figure  12  shows  a case  in  which  the  deflated  budget  curve  1 
is  modified  by  the  actual  experienced  inflation  to  derive  curve  2. 
One  may  readily  compare  this  curve  with  the  contractor's  inflated 
actuals  (curve  3)  to  determine  the  actual  extent  of  the  overrun 
(a  - b] . 


(e)  Another  situation  which  this  method  of 
program  monitoring  will  easily  handle  is  the  case  of  the  schedule 
slippage  or  program  extension.  Of  the  two,  the  slippage  is  the 
most  severe  because  it  often  occurs  early  in  the  project  where  it 
has  a profound  effect  on  costs.  Assume  once  again  the  basic  de- 
flated budget  curve  shown  in  Figure  13  with  an  actual  deflated 
expenditure  curve  is  shown. 


It  would  appear  upon  Initial  Inspection  that  the  program  Is  running 
slightly  below  the  planned  expenditures  at  time  t.  However,  It  Is 
revealed  that  the  R & D program  Is  actually  behind  schedule,  having 
only  accomplished  the  number  of  milestones  associated  with  time  t-1. 
To  compensate  for  this  slippage,  move  curve  2 back  one  unit  from 
t to  t-1  so  that  the  actual  expenditures  are  now  shown  as  curve  3 In 
their  proper  relationship  with  the  budget  curve.  This  Is  actually 
accomplished  mathematically  by  calculating  a new  Inflection  point 
which  will  reflect  the  slippage  In  the  schedule.  This  new  point  Is 
derived  from  the  equation  for  curve  2 by  calculating  the  inflection 
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point  not  at  time  T,  the  location  of  the  original  point,  but 
rather  at  time  T + 1,  the  location  of  the  inflection  point  after 
slippage  has  occurred.  This  new  inflection  point  becomes  the 
Intercept  of  the  equation  for  the  top  half  of  the  budget  curve,  and 
the  time  values  which  are  used  to  forecast  from  the  top  half  of  the 
budget  curve  now  start  at  the  T + 1 increment  (instead  of  T)  and 
contlnuj  to  the  100%  + 1 Increment  (instead  of  the  100%  Increment). 

THE  MOST  LIKELY  COST 
(FORECASTING  USING  THE  GENERAL  CURVE) 

Forecasting  the  most  likely  cost  proceeds  in  the  same  manner 
listed  in  the  previous  section  up  to  the  point  at  which  a new 
inflection  point  is  forecast.  The  actuals  are  deflated,  converted 
to  percentages  and  plotted  in  the  same  manner,  and  the  curve  form 
to  plot  these  actuals  is  the  same  equation  type  selected  to 
describe  the  bottom  half  of  the  general  curve.  At  this  point, 
however,  the  method  of  forecasting  changes  considerably. 

Instead  of  merely  splicing  the  top  half  of  the  general  curve 
onto  the  new  bottom  curve,  the  bottom  curve  is  actually  mapped  into 
the  general  curve  framework.  This  is  accomplished  as  follows: 

(1)  Using  the  deflated  actuals  from  the  program,  fit 
whatever  curve  form  is  used  in  the  bottom  half  of  the  general 
curve  to  these  data  and  forecasts  new  value  for  cumulative  expendi- 
tures at  the  inflection  point.  This  new  value  is  found  by  substi- 
tuting the  cumulative  percent  tiise  figure  which  corresponds  to  the 
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general  curve  inflection  point  into  the  new  equation  which  was 
derived  from  the  actuals. 

(2)  Take  the  new  value  for  cumulative  expenditures 
and  let  this  value  be  equal  to  the  cumulative  percent  of  budget 
figure  which  is  associated  with  the  inflection  point  on  the  general 
curve. 

(3)  Using  the  relationships  established  in  1 and  2,  the 
top  half  of  the  general  curve  may  now  be  converted  from  cumulative 
percentage  figures  to  forecast  cumulative  expenditures  for  the 
program  being  investigated. 

This  forecasting  method  has  several  advantages: 

(1)  The  time  over  which  the  program  is  planned  to  run 
is  taken  as  a given  unless  evidence  to  the  contrary  is  discovered. 

(2)  The  lower  curve  forecast  is  mapped  into  the  general 
curve  format,  thereby  creating  a smooth  S-shaped  curve  for  the 
entire  program.  Simply  splicing  the  curves  as  is  done  with  the 
budget  curve  In  the  previous  section  will  often  create  dlscontlnu- 

/ Itles  In  the  curve. 

I 

(3)  The  forecast  which  Is  created  in  this  method  is 
based  strictly  on  the  assumption  that  expenditures  In  this  particular 
program  are  proceeding  In  the  same  manner  that  all  past  programs 
have  proceeded. 

THE  HORST  POSSIBLE  COST 

Developing  the  forecast  for  the  worst  possible  cost  is  only  a 
sMtter  of  slightly  modifying  the  previous  most  likely  cost  forecast. 
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A confidence  interval  for  the  most  likely  cost  is  calculated  by  the 
methods  covered  in  Chapter  V.  The  upper  limit  of  this  confidence 
band,  based  on  whatever  level  of  confidence  was  selected  by  the 
analyst,  will  give  the  cost  figure  that  one  can  be  X%  certain  will  not 
be  exceeded.  Coupled  with  the  most  likely  cost,  this  is  an  excellent 
management  tool . 

In  summary,  one  may  develop  three  possible  forecasts  from  the 
S-shaped  curve.  The  "spliced"  curve  forecast  using  the  program 
budget  curve  reflects  an  expenditure  of  the  lowest  possible  magnitude. 
For  this  expenditure  to  be  realized,  the  program  must  run  exactly 
as  planned  from  the  inflection  point  onward.  This  is  a highly 
unlikely  situation  if  any  Increased  expenditures  have  been  incurred 
early  in  the  program.  The  most  likely  cost  and  its  confidence 
band  which  extends  to  the  upper  confidence  limit  (or  the  worst 
possible  cost)  for  the  program  are  clearly  the  most  realistic 
forecasts.  This  is  because  the  method  of  mapping  the  new  forecast 
for  the  bottom  of  the  curve  into  the  general  curve  format  places 
the  entire  program  in  a more  legitimate,  historical  perspective. 

FORECASTING  IN  THE  AREA  OF  THE  INFLECTION  POINT 

With  the  general  methods  for  handling  data  early  in  program 
established,  the  next  area  of  Interest  is  the  forecast  which  is  made 
when  the  string  of  actual  expenditures  stretches  all  the  way  to  the 
assumed  inflection  point  in  the  program.  VAten  this  takes  place  two 
courses  of  action  are  called  for  as  the  actuals  approach  and  cross 
the  inflection  point: 
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(1)  Until  the  inflection  point  has  been  reached,  the  best 
method  of  forecasting  is  to  continue  to  develop  a new  lover  half  of 
the  curve  from  the  actuals,  and  then  to  map  this  curve  into  the 
general  curve  as  was  done  in  the  previous  sections  of  this  chapter. 

(2)  After  the  actuals  appear  to  have  crossed  the  inflection 
point  {i.e.j  a large  expenditure  in  a given  period  has  been  followed 
by  two  periods  of  decreasing  expenditures)  cne  can  proceed  in  two 
different  ways: 

(a)  If  the  actual  data  points  continue  to  fit  the 
top  half  of  the  general  curve  within  an  appropriate  confidence 
Interval,  continue  to  forecast  by  using  the  top  half  of  the  general 
curve . 

(b)  If  the  actual  data  points  are  diverging  from 
the  general  curve,  and  if  enough  actual  data  points  exist  beyond 

the  inflection  point  for  regression  analysis  to  be  used,  then  calculate 
a new  equation  for  the  top  h.alf  of  the  curve  using  whichever  equation 
form  was  appropriate  in  deriving  the  general  curve.  This  method 
presumes  that  a]t  least  5 data  points  have  occurred  past  the  inflection 
point.  It  should  be  noted  that  good  accuracy  in  generating  a new 
curve  will  not  occur  until  at  least  10  points  have  been  Identified. 

CONCLUSION 

The  general  techniques  concerning  the  use  of  inflation  in  the 
model  and  the  method  of  handling  time  slippages  are  applied  to  the 
forecasts  based  on  the  general  curve  in  exactly  the  same  manner  as 
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they  were  used  with  the  budget  curve.  The  forecasting  method 
selected  should  be  the  one  which  best  serves  the  decision  maker's 


needs . 

Since  the  general  curve  would  be  representative  of  either  a 
mix  of  weapon  systems  or  a specific  type  of  weapon  system,  either 
of  which  have  been  completed  at  some  time  in  the  past,  the  data 
around  which  this  curve  is  structured  should  be  closely  examined. 
It  will  be  readily  apparent  in  most  cases  that  these  completed 
programs  all  contained  a certain  number  of  changes  during  their 
R & D phases.  Thus,  it  should  be  assumed  that  the  final  cost 
figures  which  are  generated  from  forecasts  using  the  general  curve 
also  include  a like  number  of  anticipated  program  changes,  even  if 
those  changes  are  not  visualized  at  the  early  stage  of  the  program 
when  the  forecast  is  made. 
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CHAPTER  V 
AN  APPLICATION 

Project  B is  a weapon  system  whose  R & D phase  started  in 
year  1.  The  level  6 budget  which  was  provided  by  the  contractor 
prior  to  the  start  of  the  R & D program  contained  both  a total  cost 
figure  and  individual  figures  for  the  categories  of  engineering, 
manufacturing  and  program  management  labor.  These  deflated  dollar 
figures  were  extracted  from  the  budget  and  normalized  to  a percent 


of  total  time-percent  of  total  budget  basis  in  Table  VII.  The 


5 inflection  point  for  each  of  the  four  budget  curves  was  also 


f 

j identified  at  this  time. 


The  total  expenditure  curve  was  plotted  on  the  axis  of 
Figure  7 and  compared  for  general  rationality  with  the  confidence 
band  for  all  prior  R & D programs  shown  in  Figure  3.  The  proposed 
budget  for  project  B was  contained  within  the  confidence  band  and 
the  contractor's  proposal  was  therefore  judged  to  be  sound. 

As  a second  test,  the  total  expenditure  data  was  regressed 
against  the  project  B milestones.  The  results  of  this  test  were 
as  follows: 

adj.  - .98 

F ■ 165  Durbin  Watson  Statistic:  1.97 

t • A1 
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These  statistics  show  a significant  relationship  between  the  raile- 


r 


stones  and  the  expenditure  pattern,  and,  therefore,  confirm  the 
specific  rationality  of  this  particular  expenditure  pattern  for 
this  particular  project. 

Having  passed  these  two  important  tests,  the  budget  data  can 
be  assumed  to  be  valid  and  equations  for  the  top  and  bottom  halves 
of  both  the  total  budget  curve  and  each  of  the  labor  curves  can  be 
developed.  Exhibit  HI  shows  the  output  of  the  computer  regression 
runs  for  the  total  budget  curve,  and  from  this  output  the  equations 
for  the  bottom  and  top  halves  of  the  curve  can  be  found  to  be 
bottom  half:  Y - -.193572  + 1.88402X^  + .470083Xj^^ 
top  half:  Y - .149350  + 1.75834Xj  - .91088X^^. 

All  of  the  above  research  would  be  accomplished  prior  to  the  start  of 
the  program  itself,  and  the  program  manager,  who  has  justified  in  his 
own  mind  the  validity  of  the  budget  in  both  a general  and  a specific 
sense,  may  now  sit  back  and  await  the  arrival  of  the  actuals. 

By  the  second  quarter  of  year  three,  the  actuals  shown  in 
Table  VIII  have  been  accumulated.  Ubviously,  the  program  manager 
would  have  been  forecasting  the  future  program  costs  each  time  a 
new  set  of  actuals  arrived,  but  let  us  assume  that  this  task  is  now 
to  be  accomplished  based  on  the  latest  figures  which  have  just 
arrived.  First,  the  program  manager  fits  a quadratic  curve  form 
to  each  of  the  strings  of  actuals  representing  the  total  budget 
and  the  engineering,  manufacturing  and  program  managing  labor.  A 
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quadratic  is  chosen  for  the  equation  form  because.  In  each  case. 
It  was  this  type  of  equation  which  provided  the  best  fit  for  the 
budget  curves.  For  simplicity,  we  will  only  deal  with  the  total 
budget  curve  in  this  example,  even  though  the  program  manager 
would  be  highly  interested  in  forecasting  each  of  the  other  three 
curves  in  an  actual  situation. 

The  equation  for  the  total  budget  actuals  is  developed  by  a 
computer  regression  technique,  the  results  of  which  are  shown  in 
Exhibit  IV.  The  equation  is 

Y - .00275322  - .057272Xj^  + 4.84246Xj^^. 

Figure  14  shows  the  actual  expenditures  and  the  budget  curve  for 
comparison. 


Figure  14.  Actual  Expenditures  and  the  Proposed  Budget 
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It  appears  frjm  Figure  14  that  the  actual  total  expenditures 


are  slightly  underrunning  the  budgeted  total  expenditures. 

However,  upon  comparison  of  Tables  VII  and  VIII,  it  becomes  obvious 
that  something  is  amiss.  The  engineering,  program  management,  and 
budget  expenditures  are  all  on  schedule  or  overrunning  while  the 
manufacturing  labor  is  well  below  the  budgeted  expenditure  level. 
The  program  Is  obviously  not  proceeding  as  planned,  and  more  speci- 
fically, the  engineering  and  program  managing  efforts  are  not 
producing  the  necessary  results  for  manufacturing  to  take  place. 

A further  comparison  of  the  manufacturing  figures  In  Tables 
VII  and  VIII  Indicates  that  the  program  is  actually  about  1 period 


FORECAST  USING  THE  BUDGET  CURVE 
To  determine  the  effect  which  this  slippage  has  had  on  the 
total  budget,  we  now  forecast  the  end  cost  as  follows: 

(1)  Forecast  a new  inflection  point  from  the  lower  curve  by 
using  the  derived  equation  for  the  actuals  and  the  time  period  for 
quarter  2,  year  5 instead  of  the  original  inflection  point  (quarter 
4,  year  4).  Moving  to  the  next  data  point  in  this  manner  mathemati- 
cally adjusts  the  forecast  for  the  1 period  slippage  noted  previously 
by  adding  1 time  period  to  the  lower  curve. 

new  Inflection  point  “ where 

Y^  » .00275322  - .057272(.476)  + 4.84246(.476)^ 

Y » 1.0726779 
n 

(2)  Use  the  new  Inflection  point  to  calculate  a modified 
intercept  for  the  equation  for  the  top  half  of  the  budget  data. 

From  Exhibit  I this  curve  equation  is 

Yj.  - .149350  + 1.75834Xj^  - .91088Xj^^ 

Substituting  Y^  for  yields 

Y^  - 1.0726779  - New  intercept  + 1.75834(.476)  - .91088(.476)^ 
and  1.0726779  - 1.75834(.476)  + .91088(.476)^  - New  Intercept 
.4420916  <■  New  Intercept 

Note  that  this  value  is  calculated  at  the  same  Inflection  point  used 
in  step  (1). 

(3)  This  new  equation  for  the  upper  half  of  the  actuals 
Yj.  - .4420916  + 1.75834X^  - .91088(X^)^ 

may  now  be  used  to  calculate  the  forecast  cost  at  any  point  from  the 
Inflection  point  cn  to  the  end  of  the  program.  To  calculate  the 
end  cost  of  the  R & D program  we  would  proceed  as  follows: 
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(a)  With  no  slippage  the  R & D program  should  end  at 
the  100%  of  total  time  point.  But  with  1 period  of  slippage  the 
program  should  now  end  at  the: 

(100  + 5.6)%  point 

where  each  time  Increment  has  been  equal  to 
5.6%  of  the  total  tlma 

2 

.*.  end  cost  - * .A420916  + 1.75834(1.056)  - .91088(1.056) 

= 1.283143 

or.  In  other  words,  a 1 period  slippage  In  the  program 
has  induced  a 28%  overrun  in  the  final  cost. 

FORECAST  USING  THE  GENERAL  CURVE 
Assuming  once  again  that  1 period  of  slippage  had  been 
experienced  in  the  program,  the  general  curve  may  be  used  to  forecast 
a new  end  cost  as  follows : 

(1)  The  new  inflection  point  is  forecast  in  the  same  manner 

as  before  using  the  equation  generated  from  the  actuals  in  Exhibit  IV 

Y - .00275332  - .057272( .476)  + 4 .84246( .476) ^ 
n 

or  Y - 1.0726779 
n 

(2)  This  point  is  now  mapped  into  the  framework  of  the  general 
curve.  Figure  3 shows  that  the  inflection  point  of  the  general  curve 
is  located  at  46%  cumulative  time  and  56%  cumulative  expenditures. 
Thus , if 

Y - 1.0726779,  then 
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1.0726779  » 56%  (E) 


where  E ■ total  program  expenditures. 


.*.  E - (1.0726779)  4 .56  - 1.9153 

and  the  most  likely  total  program  cost  is  forecast  to  be  191% 
of  the  budgeted  cost. 

(3)  One  can  use  the  method  outlined  in  (2)  to  fill  in  the  rest 
of  the  points  along  the  new  forecast  cost  curve,  but  it  is  interesting 
to  note  that  this  forecast  calls  for  a much  higher  figure  than  that 
produced  by  the  budget  curve  "splicing"  method. 


THE  CONFIDENCE  INTERVAL 

To  use  this  forecast  In  Its  point  estimate  format  would  be 
an  error.  Rather  there  is  a confidence  interval  around  this  fore- 
cast which  must  be  considered.  The  calculation  of  this  interval  is 
a two-part  process  dealing  first  with  the  standard  error  of  the 


forecast  for  the  new  Inflection  point.  Exhibit  IV  shows  that  S 


yx 


the  variance,  is  .000132813  for  this  particular  line.  The  standard 

error  of  the  forecast  is  calculated  from  the  formula 

:2 


f - , 


where  is,  in  this  case,  the  Inflection  point, 
for  this  example 


S-  - .0114977/1  -f  1 + .476 
' ^ 5 .0339 

• .044 

and  thus  one  can  be  95X  confident  that  the  inflection  point  is  within 
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an  interval  of 


r 


1.0726  + t , S, 

— n-1  f 

or  1.0726  + 2.776  (.044) 
1.0726  + .1246 


i 

I 


Figure  16  shows  this  confidence  band  around  the  lower  half  of  the 
curve . 

“/fa  £ » Pe u D I Tuee i 


Figure  16.  The  Confidence  Band  for  the  Lower  Half 


of  the  Actuals  Curve 

When  the  top  half  of  the  curve  la  spliced  to  the  forecast  inflection 

point  an  additional  error,  naoely  that  of  fitting  the  upper  half  of 

the  curve  to  the  budget  data,  is  encountered.  This  fitting  error 

can  be  read  from  Exhibit  III  as'S^  ■ .0000271942. 

yx 

A conservative  estimate  for  the  total  error  involved  in  the 
forecast  of  the  final  cost  can  be  gained  by  combining  the  forecast 


i 


1 

k 

I 


) 
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error  at  the  Inflection  point  with  the  fitting  error  of  the  upper 

half  of  the  curve  as  follows: 

S.  + S ■ Total  standard  error  «• 
f yx  t 

.0440  + .0052  - .0492  - S^. 

Thus  the  confidence  band  around  Che  forecast  for  the  end  cost  of 

the  program  using  Che  budget  curve  becomes 

1.283143  + t ,(SJ 
— n— 1 c 

1.283143  + (2. 776) (.0492) 

1.28  + .1366 

And,  one  can  be  95%  confident  that  the  final  cost  of  program  B 
will  be  between  114.65  and  141.98  percent  of  the  initially  budgeted 
system  cost.  Figure  17  shows  this  completed  forecast. 

% re1»i 
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Similarly,  this  process  can  be  completed  for  the  forecast  made 


with  the  budget  curve  to  derive  the  most  likely  cost.  Here  the 
standard  error  of  the  forecast  remains 


* .044 


as  has  previously  been  calculated.  However,  as  might  be  expected, 

considerably  more  error  Is  involved  in  fitting  the  top  half  of  the 

general  curve  than  was  present  In  the  budget  curve.  For  this  curve, 

composed  of  numerous  system  types.  Exhibit  II  shows  that  the  fitting 
2 

error  (S  ) is 
yx 

- .00746826 
yx 

therefore  S » .086 
yx 

and  the  total  standard  error  (S^)  Is 


S,.  - S,  + S 
t f yx 


- .044  + .086 


.13 


Now  the  confidence  Interval  for  the  most  likely  cost  Is 


1.9153  + t ,(S^) 

— n—i  t 

1.9153  + (2.776)(.13) 

1.9153  + .36 


and  there  is  95X  confidence  that  the  most  likely  cost  of  program  B Is 
between  228  and  156  percent  of  the  Initially  budgeted  system  cost. 

Another  output  of  this  forecast  is  the  worst  possible  cost. 

This  cost  Is  the  upper  limit  of  the  confidence  Interval  or.  In  this 
case,  228Z  of  the  amount  Initially  budgeted  for  the  project. 

If  these  forecasts  seem  unreasonably  high.  It  should  be  noted 
that  program  B was  an  actual  program  and  chat  the  final  cost  for  the 


R & D portion  of  program  B was  quite  close  to  the  worst  possible  cost 
(228%)  figure.  Note  also  that  this  entire  process  has  been  accom- 
plished with  deflated  data.  Incorporation  of  the  forecast  Inflation 
rates  for  this  period  would  have  greatly  Increased  the  size  of  the 
confidence  band  in  Figure  17  and  hence,  greatly  decreased  the 
possible  forecasting  accuracy  of  the  overrun. 
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APPENDIX  I 


LOGISTICS  AND  GOKPERTZ  CURVE  EQUATIONS 


The  equation  for  the  Logistic  Curve  Is 

_L 

Y ■ k + ab*  where 
c 

fr  V r difference  between 

a - _ 1)2  “ y at  X and  Y at  K 

c o c 

b - ^2^ 

/ ' ; Y T.  Y " ratio  between  successive 

•2*  1 Increments  of  growth. 

X 

k “ — (Ij^Y  - a)  * the  asymptote  or  limit 


and  Ej,  E^,  E^  refer  to  the  first,  second  and  third  parts  of 
the  Y values. 

n Is  the  number  of  observations  in  each  of  the  three  "parts" 
or  sections  of  Y values. 

The  equation  for  the  Gompertz  curve  is 

Y " k using  the  same  formulas  for  k,  a and  b as  shown 
c a ® 

above . 


47 


APPENDIX  II 


REMOVING  AUTOCORRELATION 


Given  that  autocorrelation  exists,  the  observed  residual  term 
is  actually  a combination  of  two  errors 

^ “ ^^-1  ''t 

where  Is  the  true  error  term 

and  is  the  previous  residual  term. 

Since  the  true  relationship  between  e^  and  e^  p,  is  unknown,  it 
must  be  estimated.  Several  methods  are  available  for  doing  this, 
and  this  study  employs  the  following  approach; 

(a)  compute  all  of  the  n LS  residuals 


(b)  compute  the  ratio  of  the  mean  product  of  successive 
residuals  to  the  LS  variance  estimator 


or  P 


1 lee.- 

— . a a+i 

n-1  0*1 


n-k  I e 


the  estimated  relationship 


*o«a+l 


(n-1)  S2 


Once  thia  catlaatlon  has  been  made,  3 nay  be  used  with  the 
Generalized  Least  Squares  laethod  to  remove  the  autocorrelation  from 


the  data.  This  method  proceeds  as  follows; 

Given:  Y ■ XS  + e 

where  e Is  the  matrix  of  residuals 
In  which  autocorrelation  Is  present. 
Thus  for  any  e^ 


e^  * pe 

substituting  p for  p 
matrix 


t-1  t 


, we  multiply  by  a 


"differencing" 


(3) 


(2)  may  now  be  transformed  by  (3)  to  yield 


(DY)  - (DX)6  + (De) 


(A) 


where 


De  » 


r 1 

- 

ii-i^ 

CM 

> 

-0.J  + .3 

e 

• 

m 

''3 

• 

• 

• 

e 

• 

t 

• 

''2 

the  set  of  true  error  terms. 


(5) 
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p- 


Similarly 


1/1-P 

-ayi  + y2 


px^  +X2 


-"Vl-^yn 


and  DX  = 


- px  , + X 

n-1  n 


Recalling  that  S - (X'X)"^X'Y, 
the  OLS  solution  of  (4)  is  given  by 

substituting  DY  for  Y and  DX  for  X in  (7)  to  yield 
B - t(DX) '(DX)]“^  (DX)'DY 
- [X'(D'D)X]“^  X’(D’D)Y 


From  (3), 


* , ,■'2  •“ 
-p  1+p  -p 


n - D'D 


A - ^ 

-P  1 -P 


and  the  GLS  solution  Is 


g - (x*nx)~^  x’nv  which, 

by  (5)  eliminates  the  autocorrelation  if  p=p. 


r 

since  p Is  an  estimate,  and  not  an  exact  value  for  p.  It  is  unlikely 
that  this  process  will  remove  all  autocorrelation  on  the  first 
Iteration.  For  this  reason,  multiple  iterations  are  generally 
used  with  this  technique  and  a new  P is  calculated  by  equation  (1) 

I 

I after  each  iteration.  For  n iterations 
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TABLE  IV 


0166  .865  ( 4.65)  21.64  2.39  .838  .0208 


TABLE  VI 


INFLECTION  POINT  LOCATIONS 


% Expenditures 
or 

% Milestones  % Time 


F-105  Milestones 

C-141  Milestones 

C-5  Non-Recurring 

XB-70 

Tug  R 

Tug  C 

Tug  E 

Tug  G 

B-1  Engineering  Labor 

B-1  P & M Labor 

B-1  Total  Labor 

B-1  Manufacturing  Labor 

C-5  Quality  Assurance  Hours 

C-5  Production  Hours 

A- 10  System 

A- 10  Engine 

A- 10  Gun 

A-10  Milestones 

AGM-65  A 

AWACS 


5573 

.3594 

5814 

.3400 

5385 

.5050 

5155 

.3400 

4659 

.4431 

5316 

.5500 

5026 

.4600 

6394 

.5050 

6626 

.5050 

5169 

.3590 

,4681 

.3590 

4218 

.3010 

,5866 

.3590 

.4363 

.4616 

,5246 

.4769 

,4934 

.4741 

6899 

.6245 

,5582 

.4500 

,5388 

.4205 

,6221 

.4343 

.7011 

.5050 

Mean  Inflection  Point  Locations:  .562  .462 

[The  four  B-1  labor  categories  are 
not  Included  In  these  calculations 
to  avoid  duplication  of  data.] 


Standard  Error: 

"Y  »0g52043  - y .0053252 

- .07297*4  - lo^ 


S 


expenditures 


yiH^ 


•y  .0*66686  -y 


.002918 


S - .05*0189  - lo 
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TABLE  VII 


PROGRAM  B DATA 


PERCENTAGE  OF  LEVEL  6 PROPOSED  BUDGET  FIGURES 


Q/YEAR 

TOTAL  TIME 

TOTAL  BUDGET 

ENG. LABOR 

MFG. LABOR 

P.M. LABOR 

2/1 

.0100 

.001206 

.001905 

.000015 

.001322 

4/1 

.0680 

.025086 

.045683 

.001122 

.058681 

2/2 

.1260 

.070831 

.111638 

.019189 

.148909 

4/2 

.1850 

.158547 

.199698 

.101142 

.229059 

2/3 

.2430 

.284803 

.313899 

.251838 

.315084 

4/3 

.3010 

.421767 

.414406 

.407334 

.390737 

2/4 

.3590 

.557690 

.5169<'.5 

.586607 

/CO. / o Inflection 
.468142  _ . ^ 
Points 

4/4 

.4180 

.668184 

.611508 

.722934 

.533521 

2/5 

.4760 

.759586 

.690111 

.83054 

.597940 

4/5 

.5340 

.833228 

.759773 

.925338 

.661209 

2/6 

.5920 

.874918 

.818965 

.955615 

.722603 

4/6 

.6510 

.908006 

.872539 

.965701 

.786322 

2/7 

.7090 

.937340 

.914294 

.974886 

.849762 

4/7 

.7670 

.960294 

.947851 

.983690 

.902436 

2/8 

.8250 

.978752 

.974348 

.992814 

.940941 

4/8 

.8840 

.991546 

.990384 

.998511 

.970686 

2/9 

.9420 

.993368 

.998021 

.999866 

.991682 

4/9 

1.000 

.999999 

.999999 

.999866 

.999999 

KIT:  Parcantages  in  blocka  ara  inflaction  points. 


TABLE  VIII 


PROGRAM  B 

PERCENTAGE  OF  ACTUAL  FIGURES 


ENG. 

MFG. 

0 

0 

.019 

.0006 

.074 

.0066 

.174 

.038 

.282 

.105 

P.M. 

BUDGET 

0 

0 

.036 

.024 

.101 

.080 

.196 

.145 

.300 

.280 
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GENERAL  CURVE — BOTTOM  HALF 
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GENERAL  CURVE— TOP  HALF 


BUDGET  CURVE-BOTTOM  HALF 


BUDGET  CURVE — TOP  HALF 


M&UkSIJx  k'dlLs  riTtfl  Tu' 


ACTUALS—BOTTOM  HALF 


